http://www.welcometosheffield.co.uk/visit

• Mark Dunning (Bioinformatics Core), Emily Chambers (Bioinformatics Core), Aya Elwazir (PhD Student)
• Niamh Errington (PhD student), Jon Griffin (PhD Student)

• Session 1; Data management and spreadsheets
• Session 2; R introduction and importing spreadsheets
• Session 3; Manuipulating tables and plotting
• Session 4; More plotting and summarising
• Session 5; statistics (t-tests, non-parametric tests and ANOVA)

• Live teaching / demonstrating
  • Recorded and available afterwards
• Exercises in breakout groups
• Reviewing videos and completing exercises

• Bioinformatics
  • RNA-seq, Sequencing alignment etc
• Machine Learning, A.I. etc
• You will not become an R expert after the course
  • you will hopefully feel confident to start your own analysis and how to get help

• Facebook,
• google,
• Microsoft (who recently invested in a commerical provider of R)
• The New York Times.
• Buzzfeed use R for some of their serious articles and have made the code publically available
• The New Zealand Tourist Board have R running in the background of their website
• The BBC makes code available for some of their stories (e.g. gender bias in music festivals)
• Airbnb

• The Rstudio environment
• Importing data from a spreadsheet
• Filtering Data
• Plotting
• Calculating numerical summaries
• Reporting
• Joining data from multiple spreadsheets
• Statistical Analysis

• We will cover a very small set of packages - the tidyverse

Image Credit: Aberdeen Study Group

• Spreadsheets are a common entry point for many types of analysis and Excel is used widely but
  • can be unwieldy and difficult to deal with large amounts of data
• error prone (e.g. gene symbols turning into dates)
• tedious and time consuming to repeatedly process multiple files
• how can you, or someone else, repeat what you did several months or years down the line?

• We will use data from the gapminder project

• By the end of the course we will be creating plots like this in a few lines of code

• At some point in the future, someone, somewhere, might want to repeat your analysis for themselves or re-use your data.
  • which will most likely be you!
• Assuming that you’ll be able to remember all the steps involved is dangerous, so making sure that everything is well-documented is key.

• Build research profile
• Demonstrate validity of results
• Contribute to the community
• Because you have to!
• See Florian Markowetz Self Selfish Reasons to Work Reproducibly
  • https://www.youtube.com/watch?v=yVT07Sukv9Q

• Domain-specific repository is best
• https://www.re3data.org/ Registry of Research Data Repositories
  • Gene Expression Omnibus
  • ArrayExpress
  • European Genome Archive
• Your website
• Figshare, Zenodo, institutional repository
• Have a data management plan in place from the start

Fire at CRUK Manchester - April 2017

• In your breakout groups discuss
  • what kind of research data do you have?
  • what backup strategies do you have?
  • how do you share files / data with collaborators?

Credit: CRUK_CI Bioinformatics Core

Credit: CRUK_CI Bioinformatics Core

Credit: PhD Comics

Credit: CRUK_CI Bioinformatics Core

• Consider learning about git and github for version control
• Automatic tracking of changes, person that made the changes and the reasons for it
• Interacts nicely with RStudio
• For more, see here:- Friendly Github Intro

Credit: CRUK_CI Bioinformatics Core

Credit: CRUK_CI Bioinformatics Core

• What the wet-lab scientists wanted us to spend time looking at

• What we actually spent time looking at (Disclaimer: not actual data from this project)

No matter how much of the analysis is automated, some manual steps are inevitably involved

• An analysis needs to be automated to be properly reproducible
• Tools like R, Python, will help with this but the data needs to be formatted correctly in the first place
  • R, Python cannot magically read any type of file you give them
• Making spreadsheets tidy at the start of the analysis will save a lot of time in the long-run

• ….Not neccesarily.
• Often much more convenient to eye-ball a spreadsheet and get an overall impression of your data.
• But they have limitations making them not ideal for large-scale analyses.
• Doing things by-hand only invites you to make copy-and-paste errors etc

• Never work directly on the raw data
• Hard to reverse all the manual steps performed and invites errors
• Store the original data somewhere safe
  • see later

• Consistency: F, female, f, fem, 2, …
• Units
  • cm or mm; days, months or years
• You can introduce inconsistencies without realising it
  • blank spaces (whitespace) at the end of text
  • “Male” is not the same as “male”
• Document choices you make about units in a README file

credit: @myusuf3

Figure showing locations of visitors to my Prostate Cancer data portal

• Zero values are data!
  • Sometimes extreme values such as 999 are sometimes used
• NA is Ok, but what if NA is a valid category in your data?
  • R will recognise NA as a missing value and can ignore it in calculations
• Safest to leave the cell empty
  • but you need to be careful with blank spaces

Fill in all the cells

• It is tempting to make the table look cleaner by not repeating some values
• Fill in all cells!
  • otherwise, problems when sorting
• Empty cell:
  • missing value?
  • value meant to be repeated multiple times?
• Make sure it’s clear that the data is missing and not unintentionally left blank

Make it rectangle

• The computer expects a very rigid shape of data with rows and columns
• Each column is a variable being examined
• Each row is an observation
• A concept commonly known as tidy data

• Don’t put too much information in one cell
  • 1 cell = 1 piece of information
• Don’t include units such as “30 g” → “g” in the column name
  • http://unitsofmeasure.org/ucum.html
• Write notes in a separate column or data dictionary or metadata
  • “0 (below threshold)”

• NO calculations
• NO font colours
• NO highlighting

Computer doesn’t recognize it!

• Excel data validation feature
• Select a column
  • In the menu bar, choose “Data”
  • Validation
• Integer or decimal number
• Range
• List of possible values
• Limited length text

• When identifiers are long integers
  • 1000000 = 1e06
  • Issue with Illumina microaray chip IDs
• Excel can convert gene names to dates
  • SEPT2 (Septin 2) → ‘2-Sep’
  • MARCH1 (Membrane-Associated Ring Finger (C3HC4) 1, E3 Ubiquitin Protein Ligase) → ‘1-Mar’

• Download the file patient-data.csv and open in Excel, or equivalent software
  • this is a simulated, but representative, example of bad data
  • discuss with the members of your breakout group what aspects of the dataset would make it difficult for a computer to process (around 10 minutes)
• The next step is to look at how to clean the data with Open Refine